<html>
<head><meta charset="utf-8"><title>perfbot · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html">perfbot</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="221243200"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243200" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243200">(Dec 30 2020 at 20:18)</a>:</h4>
<p>I'm afraid I caused the perfbot to hang (see <a href="https://perf.rust-lang.org/status.html">https://perf.rust-lang.org/status.html</a>). It's chewing on a 30s test since 90 minutes</p>



<a name="221243214"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243214" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243214">(Dec 30 2020 at 20:18)</a>:</h4>
<p>can someone kill it?</p>



<a name="221243440"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243440" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243440">(Dec 30 2020 at 20:21)</a>:</h4>
<p>Taking a look</p>



<a name="221243571"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243571" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243571">(Dec 30 2020 at 20:24)</a>:</h4>
<p>my guess is your code is has like n^3 or something</p>



<a name="221243781"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243781" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243781">(Dec 30 2020 at 20:27)</a>:</h4>
<p><span class="user-mention" data-user-id="124288">@oli</span> killed</p>



<a name="221243790"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243790" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243790">(Dec 30 2020 at 20:27)</a>:</h4>
<p>thanks</p>



<a name="221243797"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243797" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243797">(Dec 30 2020 at 20:27)</a>:</h4>
<p>I think my code is stuck in an infinite loop</p>



<a name="221243808"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243808" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243808">(Dec 30 2020 at 20:27)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/242791-t-infra/topic/perfbot/near/221243571">said</a>:</p>
<blockquote>
<p>my guess is your code is has like n^3 or something</p>
</blockquote>
<p>better than rustdoc's n^4 for blanket impls <span aria-label="upside down" class="emoji emoji-1f643" role="img" title="upside down">:upside_down:</span></p>



<a name="221243812"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243812" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243812">(Dec 30 2020 at 20:27)</a>:</h4>
<p>I've had that before <span aria-label="grinning" class="emoji emoji-1f600" role="img" title="grinning">:grinning:</span></p>



<a name="221243920"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243920" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243920">(Dec 30 2020 at 20:29)</a>:</h4>
<p>big-o notation is not to be treated as a <em>high score</em> <span aria-label="eyes" class="emoji emoji-1f440" role="img" title="eyes">:eyes:</span></p>



<a name="221243947"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221243947" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221243947">(Dec 30 2020 at 20:29)</a>:</h4>
<p>just pretend this is golf, small numbers are better</p>



<a name="221246298"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221246298" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221246298">(Dec 30 2020 at 21:03)</a>:</h4>
<p>looks like it got stuck again</p>



<a name="221246333"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221246333" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221246333">(Dec 30 2020 at 21:03)</a>:</h4>
<p>I promise, no more perf runs on the inliner opt without first having done it locally</p>



<a name="221246839"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221246839" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221246839">(Dec 30 2020 at 21:10)</a>:</h4>
<p>Maybe there should be a max time per benchmark? If your code takes 300x as long you probably don't want to merge it anyway lol</p>



<a name="221247346"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221247346" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221247346">(Dec 30 2020 at 21:18)</a>:</h4>
<p>Yeah we should do it timeouts are hard</p>



<a name="221247382"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221247382" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221247382">(Dec 30 2020 at 21:19)</a>:</h4>
<p><span class="user-mention" data-user-id="124288">@oli</span> where are you seeing it again?</p>



<a name="221247411"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221247411" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221247411">(Dec 30 2020 at 21:20)</a>:</h4>
<p>if you mean the rustc line that's just because we apparently don't check the "success" when recording previous time</p>



<a name="221247430"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221247430" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221247430">(Dec 30 2020 at 21:20)</a>:</h4>
<p>ah it just moved on... rustc took like 40x as much time</p>



<a name="221247457"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/221247457" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#221247457">(Dec 30 2020 at 21:20)</a>:</h4>
<p>i.e. that's from a failing compile</p>



<a name="229630033"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229630033" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> llogiq <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229630033">(Mar 10 2021 at 09:53)</a>:</h4>
<p>stealing this topic to discuss a suggestion I made on twitter: We should add a suite of UI tests to the perf bot. For now, the speed of <em>failing</em> builds isn't tracked at all, and we should have UI tests for most if not all possible errors. I could imagine that we may want to track the different output formats as well as possibly check vs. buid, but I'm unsure iff the latter really makes any difference.</p>



<a name="229668169"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229668169" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229668169">(Mar 10 2021 at 14:25)</a>:</h4>
<p>Why would we care about the speed of a failing build?</p>



<a name="229713497"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229713497" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> llogiq <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229713497">(Mar 10 2021 at 18:04)</a>:</h4>
<p>Because while developing, builds fail more often than not, and thus, the time saved with each perf boost could amortize even more than with succeeding ones.</p>



<a name="229715115"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229715115" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229715115">(Mar 10 2021 at 18:14)</a>:</h4>
<p>Hmm, I think the metric there should be "time to first error", not the total time to run</p>



<a name="229715243"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229715243" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229715243">(Mar 10 2021 at 18:15)</a>:</h4>
<p>And that varies wildly depending on what the error is</p>



<a name="229766456"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229766456" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229766456">(Mar 10 2021 at 23:31)</a>:</h4>
<p>E.g., name resolution errors show up <em>really</em> fast—in just 5 minutes for <a href="https://github.com/rust-lang/rust/pull/82993/checks?check_run_id=2081786831">https://github.com/rust-lang/rust/pull/82993/checks?check_run_id=2081786831</a> (which had an error late in the build, in rustdoc).</p>



<a name="229813034"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229813034" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> llogiq <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229813034">(Mar 11 2021 at 08:36)</a>:</h4>
<p>Well, there's also "time to last error", which may also be of interest, but both aren't that easy to measure. So I'll take "time to run" as a workable approximation.</p>



<a name="229990932"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/perfbot/near/229990932" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> llogiq <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/perfbot.html#229990932">(Mar 12 2021 at 08:10)</a>:</h4>
<p>To clarify, we may well want to get all those timings (time to first output, time to last output, time to completion), but time to completion is the easiest to get, so I propose we start there.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>